Computer assisted identification of appropriate anatomical structure for medical device placement during a surgical procedure

ABSTRACT

A method for computer assisted identification of appropriate anatomical structure for placement of a medical device, comprising: receiving a 3D scan volume comprising set of medical scan images of a region of an anatomical structure where the medical device is to be placed; automatically processing the set of medical scan images to perform automatic segmentation of the anatomical structure; automatically determining a subsection of the 3D scan volume as a 3D ROI by combining the raw medical scan images and the obtained segmentation data; automatically processing the ROI to determine the preferred 3D position and orientation of the medical device to be placed with respect to the anatomical structure by identifying landmarks within the anatomical structure with a pre-trained prediction neural network; automatically determining the preferred 3D position and orientation of the medical device to be placed with respect to the 3D scan volume of the anatomical structure.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/537,645, filed Aug. 12, 2019, entitled “Computer AssistedIdentification of Appropriate Anatomical Structure for Medical DevicePlacement During a Surgical Procedure,” the disclosure of which ishereby incorporated by reference.

U.S. patent application Ser. No. 16/537,645 claims priority to and thebenefit of EP Patent Application No. 18188557.5, filed Aug. 10, 2018,entitled “Computer Assisted Identification of Appropriate AnatomicalStructure for Medical Device Placement During a Surgical Procedure.”

TECHNICAL FIELD

The invention relates to computer assisted surgical navigation systems,in particular to a system and method for identifying appropriateanatomical structure for placement of a medical device, such asinstrumentation or implant, during a surgical procedure, in particularrelated to neurological and general surgery procedures.

BACKGROUND

Image guided or computer assisted surgery is a surgical procedure wherethe surgeon uses trackable surgical instruments, combined withpreoperative or intraoperative images (e.g., from computed tomography(CT) scanners), in order to provide the surgeon with surgical guidanceduring the procedure.

SUMMARY OF THE INVENTION

One of the disadvantages of known methods of image guided or computerassisted surgery is that they are not fully automatic. They require aspecialized person to analyze the X-Ray, CT or NMR data and select astarting point for the procedure. Moreover they do not mention anythingabout the intraoperative CT allowing proper positioning during thesurgery. In contrast, the invention, in certain embodiments, allows forfully automatic positioning and size determination in the 3D domain ofthe ongoing surgery thanks to usage of an intraoperative scanner andArtificial-Intelligence-based methods.

One aspect of the invention is a method for computer assistedidentification of appropriate anatomical structure for placement of amedical device, comprising: receiving a 3D scan volume comprising set ofmedical scan images of a region of an anatomical structure where themedical device is to be placed; automatically processing the set ofmedical scan images to perform automatic segmentation of the anatomicalstructure; automatically determining a subsection of the 3D scan volumeas a 3D region of interest by combining the raw medical scan images andthe obtained segmentation data; automatically processing the ROI todetermine the preferred 3D position and orientation of the medicaldevice to be placed with respect to the anatomical structure byidentifying landmarks within the anatomical structure with a pre-trainedprediction neural network; automatically determining the preferred 3Dposition and orientation of the medical device to be placed with respectto the 3D scan volume of the anatomical structure.

The method may further comprise automatically identifying and storingthe 3D position and orientation of the medical device placed by thesurgeon in the anatomical structure during the surgical procedure, andusing this information for further training of the prediction neuralnetwork in order to improve accuracy of the prediction neural network tosubsequently identify the preferred positions and orientations to besuggested to the surgeon in successive surgical procedures.

The method may further comprise processing the scan images of theanatomical structures between the identified landmarks, and determiningphysical dimensions of the anatomical structures in the region ofinterest where the medical device is intended to be placed.

The method may further comprise determining preferred physicaldimensions, the preferred physical dimensions including at least one ofsize, diameter and length, of the medical device to be placed dependingon analyzed dimensions of the anatomical structure.

The received medical scan images may be collected from an intraoperativescanner.

The received medical scan images may be collected from a presurgicalstationary scanner.

Another aspect of the invention is a computer-implemented system,comprising: at least one nontransitory processor-readable storage mediumthat stores at least one of processor-executable instructions or data;and at least one processor communicably coupled to at least onenontransitory processor-readable storage medium, wherein at least oneprocessor is configured to perform the steps of the method as describedherein.

These and other features, aspects and advantages of the invention willbecome better understood with reference to the following drawings,descriptions and claims.

BRIEF DESCRIPTION OF DRAWINGS

Various embodiments are herein described, by way of example only, withreference to the accompanying drawings, wherein:

FIG. 1 shows an overview of a training procedure in accordance with anembodiment of the invention;

FIG. 2A shows an image used in the system during the trainingprocedures, in accordance with an embodiment of the invention;

FIG. 2B shows an image used in a system during the training procedures,in accordance with an embodiment of the invention;

FIG. 2C shows an image used in the system during the trainingprocedures, in accordance with an embodiment of the invention;

FIG. 2D shows region of interest used in the process, in accordance withan embodiment of the invention;

FIG. 2E-1 shows 3 dimensional resizing of region of interest, inaccordance with an embodiment of the invention;

FIG. 2E-2 shows 3 dimensional resizing of region of interest, inaccordance with an embodiment of the invention;

FIG. 2F shows exemplary characteristic features localization, inaccordance with an embodiment of the invention;

FIG. 2G shows exemplary results of artificial training databaseaugmentation, in accordance with an embodiment of the invention;

FIG. 2H shows exemplary final implant localization, in accordance withan embodiment of the invention;

FIG. 3 shows an overview of a prediction procedure, in accordance withan embodiment of the invention;

FIG. 4 shows a prediction CNN architecture, in accordance with anembodiment of the invention;

FIG. 5 shows a flowchart of a training process for the prediction CNN,in accordance with an embodiment of the invention;

FIG. 6 shows a flowchart of an inference process for the prediction CNN,in accordance with an embodiment of the invention; and

FIG. 7 shows the structure of a computer system for implementing themethod of FIG. 1, in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The following detailed description is of the best currently contemplatedmodes of carrying out the invention. The description is not to be takenin a limiting sense, but is made merely for the purpose of illustratingthe general principles of the invention.

The term “medical device” as used herein is understood to mean asurgical implant or an instrument, for example a catheter, instrument, acannula, a needle, an anchor, a screw, a stent, a biomechanical device.

The invention is described below in detail with reference to anembodiment related to a neurological surgery, wherein a screw (as anexample of a medical device) is placed, i.e. inserted, to a spine (as anexample of an anatomical structure). A skilled person will realize thatthis embodiment can be extended to other applications as well, such asguidance for a medical device (e.g., instrumentation or implant) inother natural or artificial anatomical structures, for example bloodvessels, biliary ducts, subthalamic nucleus, and components of solidorgans like the heart (e.g., mitral valve), kidney (e.g., renal artery),and nerves (e.g., epidural space).

The automatic implant placement method as presented herein comprises twomain procedures: a training procedure and a prediction procedure.

In certain embodiments, the training procedure, as presented in FIG. 1,comprises the following steps. First, in step 101, a set of DICOM(Digital Imaging and Communications in Medicine) images obtained with apreoperative or an intraoperative CT (Computed Tomography) or MRI(Magnetic Resonance Imaging) representing consecutive slices withvisible tissues is received (such as one slice shown in FIG. 2A). Next,the received images are processed in step 102 to perform automaticsegmentation of tissues, such as to determine separate areascorresponding to different tissues, such as vertebral body 16, pedicles15, transverse processes 14 and/or spinous process 11, as shown in FIG.2B. For example, this can be done by employing a method for segmentationof images disclosed in Applicant's European patent applicationEP17195826 filed Oct. 10, 2017 and published as EP 3 470 006 A1 on Apr.17, 2019. Then, in step 103, the information obtained from DICOM imagesand the segmentation results is merged to obtain combined imagecomprising information about the tissue appearance and itsclassification (including assignment of structure parts to classescorresponding to different anatomy parts), for example in a form of acolor-coded DICOM image, as shown in FIG. 2C. Alternatively, separateDICOM (FIG. 2A) and segmentation (FIG. 2B) images can be processedinstead of the combined image. Next, in step 104, from the set of sliceimages a 3D region of interest (ROI) 18 is determined, that contains avolume of each pedicle 15 with a part of adjacent vertebral body 16 andsurrounding tissues such as lamina 13, transverse process 14 and others,as shown in FIG. 2D. Then, in step 105, the 3D resizing of thedetermined ROI 18 is performed to achieve the same size of all ROI'sstacked in the 3D matrices, each containing information about voxeldistribution along X, Y and Z axes and the appearance and classificationinformation data for each voxel, such as shown in FIG. 2E-1 or 2E-2. Inother words, the voxels are small cuboidal volumes resembling pointshaving 3D coordinates and a radiodensity value and classificationassigned.

Next, in step 106, a training database is prepared manually, thatcomprises the previously determined ROIs and manually landmarkedcharacteristic features corresponding to pedicle center 25 and screw tip27 (or other anatomical structure and device points), such as shown inFIG. 2F.

Next, in step 107, the training database is augmented, for example withthe use of a 3D generic geometrical transformation and resizing withrandom dense 3D grid deformations, as shown in FIG. 2G. Dataaugmentation is performed on the images to make the training set morediverse. The foregoing transformations are remapping the voxelspositions in a 3D ROI 18 based on a randomly warped artificial gridassigned to the ROI 18 volume. A new set of voxel positions iscalculated artificially warping the 3D tissue shape and appearance.Simultaneously, the information about the tissue classification iswarped to match the new tissue shape and the manually determinedlandmarks positions 25, 27 are recalculated in the same manner. Duringthe process, the value of each voxel, containing information about thetissue appearance, is recalculated in regards to its new position in ROI18 with use of an interpolation algorithm (for example bicubic,polynomial, spline, nearest neighbor, or any other interpolationalgorithm) over the 3D voxel neighborhood.

Next, in step 108, the obtained artificial database augmentation resultsare combined with the automatically recalculated landmarks,corresponding to the artificially augmented pedicle centers 25 and screwtips 27 (or other anatomical structure and device points), into a singledatabase interpretable by a neural network.

Then, in step 109, the placement prediction model is trained with aneural network. In certain embodiments, a network with a plurality oflayers is used, specifically a combination of convolutional and fullyconnected layers with ReLU activation functions or any other non-linearor linear activation functions. For example, a network such as shown inFIG. 4, according to a process such as shown in FIG. 5, can be used.

The training database may also comprise data from actually performedsurgical procedures. The system may automatically identify and store the3D position and orientation of the medical device actually inserted bythe surgeon in the anatomical structure during the surgical procedure,for further training the prediction neural network (400) in order toimprove its performance to subsequently identify the preferred positionsand orientations. Therefore, the system may operate like a closedfeedback loop.

In certain embodiments, the prediction procedure, as presented in FIG.3, comprises the following steps. First, in step 301, a 3D scan volumeis received, comprising a set of DICOM (Digital Imaging andCommunications in Medicine) images of a region of the anatomicalstructure where the medical device is to be placed. The 3D scan volumecan be obtained with a preoperative or an intraoperative CT (ComputedTomography) or Mill (Magnetic Resonance Imaging). The set of DICOMsrepresenting consecutive slices of a spine is received (such as oneslice shown in FIG. 2A). Next, the received images are processed in step302 to perform automatic segmentation of tissues of the anatomicalstructure, such as to determine separate areas corresponding todifferent tissues, such as vertebral body 16, pedicles 15, transverseprocesses 14, lamina 13 and/or spinous process 11, as shown in FIG. 2B.For example, this can be done by employing a method for segmentation ofimages disclosed in Applicant's European patent application EP17195826filed Oct. 10, 2017 and published as EP 3 470 006 A1 on Apr. 17, 2019,incorporated herein by reference in its entirety. Then, in step 303, theinformation obtained from DICOM images and the segmentation results ismerged to obtain combined image comprising information about the tissueappearance and its classification, for example in a form of acolor-coded DICOM image, as shown in FIG. 2C. Alternatively, separateDICOM (FIG. 2A) and segmentation (FIG. 2B) images can be processedinstead of the combined image. Next, in step 304, from the 3D scanvolume a 3D region of interest (ROI) 18 is automatically determined. Forexample, the ROI 18 may contain a volume of each pedicle 15 with a partof adjacent vertebral body and surrounding tissues, as shown in FIG. 2D.Then, in step 305, the 3D resizing of the determined ROI 18 is performedto achieve the same size of all ROI's stacked in the 3D matrices. Each3D matrix contains information about voxel distribution along X, Y and Zaxes with bone density and classification information data for eachvoxel, such as shown in FIG. 2E-1 or 2E-2. Therefore, steps 301-305 areperformed in a way similar to steps 101-105 of the training procedure ofFIG. 1.

Next, in step 306, the preferred placement is predicted automatically byprocessing the resized ROI to determine the preferred 3D position andorientation of the medical device to be placed with respect to theanatomical structure, by means of the pretrained prediction CNN 400,according to the prediction process presented in FIG. 6. The predictionCNN 400 is configured to identify landmarks within the anatomicalstructure, such as pedicle center 25 and screw tip 27.

Next, in step 307, the predicted screw tip 25 and pedicle center 27 (orother anatomical structure and device landmarks) positions within theROI are backward recalculated to meet the original ROI size andpositions from input DICOM dataset to recreate and ensure a correctplacement in original volume.

In step 308 the information about the global coordinate system (ROIposition in the DICOM dataset) and local ROI coordinate system(predicted screw tip and pedicle center positions inside the ROI) isrecombined.

Then, in step 309, the preferred device positioning in the 3D space iscalculated, based on two landmarks corresponding to pedicle center 25and screw tip 27, as shown in FIG. 2F.

Anatomical knowledge and preferred device positioning allow for thecalculation of a preferred device's physical dimensions, for examplescrew positioning in the vertebra. With the semantic/anatomicalsegmentation results and pedicle center 25 location available, in step310, automated computation of device physical dimensions, such as thediameter, is possible. Proceeding in the coronal direction, forward andbackward from the pedicle center landmark 25 along the pedicle, theslice for which the inscribed circle diameter will be the smallest caneasily be found. A fraction of this diameter corresponds directly to theinserted device maximum allowed diameter with a necessary safety marginthat can be easily defined by the user of the system.

Enabling selection of a specific element in the available series oftypes also requires determination of device physical dimensions such asthe length. This too can be easily computed automatically using thedevice insertion trajectory information provided by the neural network.The line going through the estimated landmarks (bone anchor tip 27,pedicle center 25) represents the trajectory of the device, which can beexpressed as a 3D path, in the case of 2 landmarks it will be linemodel. Given the trajectory of a medical device to be inserted and ananatomical structure being a target, the entry and exit points could becalculated using automated 3D image analysis. For example, given the 3Dline model and a 3D shell of the shape of the anatomical part being atarget of device insertion extracted using morphological gradient in 3D(a single voxel thick surface of all solids in the volume), the entryand exit points of the trajectory are located at the two shell voxels(XX, YY) that are closest to the line (trajectory T) at each end, forexample such as shown in FIG. 2F.

Next, in step 311, the output is visualized, for example such as shownin FIG. 2H, including the device 31 to be inserted.

FIG. 4 shows a convolutional neural network (CNN) architecture 400,hereinafter called the prediction CNN, which is utilized in certainembodiments of the method of the invention for prediction of deviceplacement. The network performs device localization task using at leastone input as a 3D information about the appearance (radiodensity) andthe classification for each voxel in a 3D ROI.

The left side of the network is a contracting path, which includesconvolution layers 401 and pooling layers 402, and the right side is aregression path which includes fully connected layers 403 and the outputlayer 404.

One or more 3D ROI's can be presented to the input layer of the networkto learn reasoning from the data.

The convolution layers 401 can be of a standard kind, the dilated kind,or a combination thereof, with ReLU, leaky ReLU or any other kind ofactivation function attached.

The fully connected layers 403 can have Linear, ReLU or any other kindof activation function attached.

The output layer 404 also denotes the fully connected layer with theloss function, for example the loss function can be implemented as meansquared error or another metric.

The architecture is general, in the sense that adopting it to ROI's ofdifferent size is possible by adjusting the size (resolution) of thelayers. The number of layers and number of filters within a layer isalso subject to change, depending on the requirements of theapplication, for example as presented in Applicant's European patentapplication EP17195826.

The final layer for the device placement defines the preferred deviceposition and orientation along X, Y and Z axes in 3D ROI. Prediction isbased on the model trained from the manually prepared examples duringthe training process, for example in case of screw insertion, preferredposition of the pedicle center 25 and screw tip 27.

FIG. 5 shows a flowchart of a training process, in accordance withcertain embodiments, which can be used to train the prediction CNN 400.The objective of the training for the prediction CNN 400 is to tune theparameters of the prediction CNN 400 such that the network is able topredict preferred guidance for the device.

The training database may be separated into a training set used to trainthe model, a validation set used to quantify the quality of the model,and a test set.

The training starts at 501. At 502, batches of training ROI's are readfrom the training set, one batch at a time.

At 503 the ROI's can be additionally augmented. Data augmentation isperformed on these ROI's to make the training set more diverse. Theinput/output data is subjected to the combination of transformationsfrom the following set: rotation, scaling, movement, horizontal flip,additive noise of Gaussian and/or Poisson distribution and Gaussianblur, volumetric grid deformation, etc. or could be augmented with theuse of generative algorithm such as Generative Adversarial Networks forexample.

At 504, the ROI's are then passed through the layers of the CNN in astandard forward pass. The forward pass returns the results, which arethen used to calculate at 505 the value of the loss function—thedifference between the desired and the computed outputs. The differencecan be expressed using a similarity metric (e.g., mean squared error,mean average error or another metric).

At 506, weights are updated as per the specified optimizer and optimizerlearning rate using Gradient Descent methods (e.g., Stochastic GradientDescent, Adam, Nadam, Adagrad, Adadelta, RMSprop).

The loss is also back-propagated through the network, and the gradientsare computed. Based on the gradient values, the network's weights areupdated. The process (beginning with the ROI's batch read) is repeatedcontinuously until the end of the training session is reached at 507.

Then, at 508, the performance metrics are calculated using a validationdataset—which is not explicitly used in training set. This is done inorder to check at 509 whether or not the model has improved. If it isnot the case, the early stop counter is incremented at 514 and it ischecked at 515 if its value has reached a predefined number of epochs.If so, then the training process is complete at 516, since the model hasnot improved for many sessions now.

If the model has improved, the model is saved at 510 for further use andthe early stop counter is reset at 511. As the final step in a session,learning rate scheduling can be applied. The sessions at which the rateis to be changed are predefined. Once one of the session numbers isreached at 512, the learning rate is set to one associated with thisspecific session number at 513.

Once the training is complete, the network can be used for inference(i.e., utilizing a trained model for prediction on new data).

FIG. 6 shows a flowchart of an inference process for the prediction CNN400 in accordance with certain embodiments of the invention.

After inference is invoked at 601, a set of ROI's is loaded at 602 andthe prediction CNN 400 and its weights are loaded at 603.

At 604, one batch of ROI's at a time is processed by the inferenceserver.

At 605, the images can be preprocessed (e.g., normalized)

At 606, a forward pass through the prediction CNN 400 is computed.

At 607, a postprocess prediction is done.

At 608, if not all batches have been processed, a new batch is added tothe processing pipeline until inference has been performed on all inputROI's.

Finally, at 609, the inference results are saved and can be recalculatedto provide an output in a form of preferred device position.

The functionality described herein can be implemented in a computersystem 700, such as shown in FIG. 7. The system 700 may include at leastone nontransitory processor-readable storage medium 710 that stores atleast one of processor-executable instructions 715 or data; and at leastone processor 720 communicably coupled to the at least one nontransitoryprocessor-readable storage medium 710. The at least one processor 720may be configured to (by executing the instructions 715) perform theprocedure of FIG. 1.

While the invention has been described with respect to a limited numberof embodiments, it will be appreciated that many variations,modifications and other applications of the invention may be made.Therefore, the claimed invention as recited in the claims that follow isnot limited to the embodiments described herein.

1.-7. (canceled)
 8. A method, comprising: receiving a set ofthree-dimensional (3D) scan volumes each including image data of ananatomical structure with a medical device implanted in the anatomicalstructure; processing each of the set of 3D scan volumes to obtainsegmentation data that identifies different anatomical parts of theanatomical structure of each of the set of 3D scan volumes; merging, foreach of the set of 3D scan volumes, the segmentation data of that 3Dscan volume with the image data of that 3D scan volume to producecombined image data for that 3D scan volume; identifying a 3D region ofinterest (ROI) in the combined image data of each of the set of 3D scanvolumes that includes a subsection of the anatomical structure of that3D scan volume with the medical device implanted therein; generating atraining database including (1) the 3D ROI identified for each of theset of 3D scan volumes and (2) marked characteristic features associatedwith a set of landmarks in the 3D ROI identified for each of the set of3D scan volumes, the set of landmarks including an anatomical part ofthe anatomical structure and a portion of the medical device implantedin the anatomical structure; and training, using the training database,a prediction neural network model to identify preferred positions andorientations for placement of the medical device in the anatomicalstructure.
 9. The method of claim 8, further comprising: augmenting thetraining database by: transforming the 3D ROI identified for each of theset of 3D scan volumes using one or more of: rotation, scaling,movement, horizontal flip, additive noise of Gaussian or Poissondistributions and Gaussian blur, volumetric grid deformation, or agenerative algorithm, and generating an augmented training databaseincluding the 3D ROIs and the transformed 3D ROIs; and training theprediction neural network model using the augmented training database.10. The method of claim 9, wherein the augmenting the training databasefurther includes recalculating a position of the set of landmarks ineach of the transformed 3D ROIs and associating the recalculatedposition of the set of landmarks with the transformed 3D ROIs.
 11. Themethod of claim 8, further comprising: resizing, for each of the set of3D scan volumes, the 3D ROI identified for that 3D scan volume such thateach 3D ROI has the same size, the training database including theresized 3D ROI for each of the set of 3D scan volumes and the markedcharacteristic features associated with the set of landmarks in theresized 3D ROI.
 12. The method of claim 8, further comprisingdetermining, using the trained prediction neural network model, apreferred placement of the medical device in an anatomical structure ofa patient.
 13. The method of claim 12, wherein the determining thepreferred placement of the medical device in the anatomical structure ofthe patient includes: determining, based on a 3D scan volume ofanatomical structure of the patient and segmentation data of theanatomical structure of the patient, a 3D ROI in the 3D scan volume ofthe anatomical structure of the patient for placement of the medicaldevice; processing, using the trained prediction neural network model,the 3D ROI of the patient to identify the set of landmarks in the 3D ROIof the patient; and determining a 3D position and orientation of themedical device for the preferred placement of the medical device basedon the set of landmarks.
 14. The method of claim 13, further comprisingvisualizing the 3D position and orientation of the medical device forthe preferred placement of the medical device with respect to theanatomical structure.
 15. The method of claim 13, further comprisingidentifying, after the medical device has been placed in the anatomicalstructure of the patient, an actual 3D position and orientation of themedical device; and further training the prediction neural network modelbased on the actual 3D position and orientation of the medical device.16. The method of claim 8, wherein the anatomical part of the set oflandmarks includes a pedicle, and the portion of the medical device ofthe set of landmarks includes a tip of the medical device.
 17. A method,comprising: receiving a set of three-dimensional (3D) scan volumes eachincluding image data of an anatomical structure with a medical deviceimplanted in the anatomical structure; processing each of the set of 3Dscan volumes to obtain segmentation data that identifies differentanatomical parts of the anatomical structure of each of the set of 3Dscan volumes; identifying a 3D region of interest (ROI) in each of theset of 3D scan volumes that includes a subsection of the anatomicalstructure of that 3D scan volume with the medical device implantedtherein; generating a training database including (1) the 3D ROIidentified for each of the set of 3D scan volumes and (2) markedcharacteristic features associated with a set of landmarks in the 3D ROIidentified for each of the set of 3D scan volumes, the set of landmarksincluding an anatomical part of the anatomical structure and a portionof the medical device implanted in the anatomical structure; training,using the training database, a prediction neural network model toidentify preferred positions and orientations for placement of themedical device in the anatomical structure; and determining, using thetrained prediction neural network model, a preferred placement of themedical device in an anatomical structure of a patient.
 18. The methodof claim 17, further comprising: augmenting the training database by:transforming the 3D ROI identified for each of the set of 3D scanvolumes using one or more of: rotation, scaling, movement, horizontalflip, additive noise of Gaussian or Poisson distributions and Gaussianblur, volumetric grid deformation, or a generative algorithm, andgenerating an augmented training database including the 3D ROIs and thetransformed 3D ROIs; and training the prediction neural network modelusing the augmented training database.
 19. The method of claim 18,wherein the augmenting the training database further includesrecalculating a position of the set of landmarks in each of thetransformed 3D ROIs and associating the recalculated position of the setof landmarks with the transformed 3D ROIs.
 20. The method of claim 17,further comprising: resizing, for each of the set of 3D scan volumes,the 3D ROI identified for that 3D scan volume such that each 3D ROI hasthe same size, the training database including the resized 3D ROI foreach of the set of 3D scan volumes and the marked characteristicfeatures associated with the set of landmarks in the resized 3D ROI. 21.The method of claim 17, further comprising determining, using thetrained prediction neural network model, a preferred placement of themedical device in an anatomical structure of a patient.
 22. Anapparatus, comprising: a memory; and a processor operatively coupled tothe memory, the processor configured to: receive a set ofthree-dimensional (3D) scan volumes each including image data of ananatomical structure with a medical device implanted in the anatomicalstructure; process each of the set of 3D scan volumes to obtainsegmentation data that identifies different anatomical parts of theanatomical structure of each of the set of 3D scan volumes; merge, foreach of the set of 3D scan volumes, the segmentation data of that 3Dscan volume with the image data of that 3D scan volume to producecombined image data for that 3D scan volume; identify a 3D region ofinterest (ROI) in the combined image data of each of the set of 3D scanvolumes that includes a subsection of the anatomical structure of that3D scan volume with the medical device implanted therein; generate atraining database including (1) the 3D ROI identified for each of theset of 3D scan volumes and (2) marked characteristic features associatedwith a set of landmarks in the 3D ROI identified for each of the set of3D scan volumes, the set of landmarks including an anatomical part ofthe anatomical structure and a portion of the medical device implantedin the anatomical structure; and train, using the training database, aprediction neural network model to identify preferred positions andorientations for placement of the medical device in the anatomicalstructure.
 23. The apparatus of claim 22, wherein the processor isfurther configured to determine, using the trained prediction neuralnetwork model, a preferred placement of the medical device in ananatomical structure of a patient.
 24. The apparatus of claim 22,wherein the processor is further configured to: determine, based on a 3Dscan volume of anatomical structure of the patient and segmentation dataof the anatomical structure of the patient, a 3D ROI in the 3D scanvolume of the anatomical structure of the patient for placement of themedical device; process, using the trained prediction neural networkmodel, the 3D ROI of the patient to identify the set of landmarks in the3D ROI of the patient; and determine a 3D position and orientation ofthe medical device for the preferred placement of the medical devicebased on the set of landmarks.
 25. The apparatus of claim 24, furthercomprising a display device communicably coupled to the processor,wherein the display device is configured to display the 3D position andorientation of the medical device for the preferred placement of themedical device with respect to the anatomical structure.
 26. Theapparatus of claim 25, wherein the processor is further configured to:identify, after the medical device has been placed in the anatomicalstructure of the patient, an actual 3D position and orientation of themedical device; and further train the prediction neural network modelbased on the actual 3D position and orientation of the medical device.27. The apparatus of claim 22, wherein the anatomical part of the set oflandmarks includes a pedicle, and the portion of the medical device ofthe set of landmarks includes a tip of the medical device.